<html>
<head><meta charset="utf-8"><title>end-to-end queries · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html">end-to-end queries</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="155002545"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002545" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002545">(Jan 12 2019 at 22:35)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> figured it'd be a good idea to slowly separate a topic for this work - is that fine?</p>



<a name="155002732"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002732" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002732">(Jan 12 2019 at 22:40)</a>:</h4>
<p>I spent some time to understand the code and slowly tried to convert more stuff on the expansion/hir/analysis boundary only to just now understand recent's <span class="user-mention" data-user-id="116466">@Zoxc</span> (e.g. allocating HIR on arena, pushing more checks to incremental infra) great work <span class="emoji emoji-1f601" title="grinning face with smiling eyes">:grinning_face_with_smiling_eyes:</span></p>



<a name="155002747"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002747" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002747">(Jan 12 2019 at 22:41)</a>:</h4>
<p>so I decided not to touch much stuff from those PRs and instead slowly chip at Session/pass-based code elsewhere</p>



<a name="155002803"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002803" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002803">(Jan 12 2019 at 22:42)</a>:</h4>
<p>I have a question - I decided to at first querify <code>Session::proc_macro_decls_static</code> which worked out okay but I found out there's a <code>proc_macro_decls_static</code> query that is defined for the external crates via cstore, in the librustc_metadata crate</p>



<a name="155002818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002818">(Jan 12 2019 at 22:43)</a>:</h4>
<p>and so I created a separate <code>proc_macro_decls_static_local</code> crate, which only works for <code>LOCAL_CRATE</code> - is there a way to combine those into a single query, where for <code>LOCAL_CRATE</code> key it traverses the AST like before but for other crate nums it tries to pull them from the externally loaded crates?</p>



<a name="155002866"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002866" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002866">(Jan 12 2019 at 22:44)</a>:</h4>
<p>or is such split still beneficial from data flow visualization/analysis point of view?</p>



<a name="155002885"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155002885" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155002885">(Jan 12 2019 at 22:45)</a>:</h4>
<p>for the context, this is my current diff: <a href="https://github.com/rust-lang/rust/commit/16bd7050b53f95882eaef0aa2f975316e969b081" target="_blank" title="https://github.com/rust-lang/rust/commit/16bd7050b53f95882eaef0aa2f975316e969b081">https://github.com/rust-lang/rust/commit/16bd7050b53f95882eaef0aa2f975316e969b081</a></p>



<a name="155005140"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155005140" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155005140">(Jan 12 2019 at 23:56)</a>:</h4>
<p>You should be able to use the <code>proc_macro_decls_static</code> query. It will dispatch to the metadata if the crate is external, otherwise it will run the regular provider</p>



<a name="155005773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155005773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155005773">(Jan 13 2019 at 00:12)</a>:</h4>
<p>oooh, since this exists on <code>Providers</code> but we can specify functions there separately either locally or via extern providers</p>



<a name="155005775"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155005775" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155005775">(Jan 13 2019 at 00:13)</a>:</h4>
<p>makes sense, thank you!</p>



<a name="155255955"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/155255955" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#155255955">(Jan 16 2019 at 14:13)</a>:</h4>
<p>Sorry <span class="user-mention" data-user-id="153740">@Igor Matuszewski</span> just saw this message, but yeah -- "what <span class="user-mention" data-user-id="116466">@Zoxc</span> said", basically =)</p>



<a name="156612443"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156612443" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156612443">(Jan 22 2019 at 15:20)</a>:</h4>
<p>I now have a concrete plan on how to bring the queries I defined in <a href="https://internals.rust-lang.org/t/making-the-compiler-interface-on-demand-driven/9099/6" target="_blank" title="https://internals.rust-lang.org/t/making-the-compiler-interface-on-demand-driven/9099/6">https://internals.rust-lang.org/t/making-the-compiler-interface-on-demand-driven/9099/6</a> into the proper query system without causing performance regressions.<br>
By using <code>no_hash</code> (added in <a href="https://github.com/rust-lang/rust/pull/57770" target="_blank" title="https://github.com/rust-lang/rust/pull/57770">https://github.com/rust-lang/rust/pull/57770</a>) and <code>eval_always</code> on the queries, we can avoid the overhead of incremental compilation. We also avoid the need to introduce <code>HashStable</code> impls.<br>
Creating the HIR map will be a single query, which also sets up the <code>Krate</code>, <code>HirBody</code> and Hir dep nodes. These will no longer be treated as inputs and forcing them will result in the HIR map query being run.<br>
We can think of this as the HIR map query producting multiple return value from the perspective of incremental compilation.<br>
Calling <code>tcx.hir()</code> will invoke the HIR map in an ignore context and return a reference to the HIR map. Accessing the HIR map will add the required read edges to the dep graph. Since the HIR map is accessed frequently, it will be cached in <code>GlobalCtxt</code> to avoid the overhead of the query system.</p>
<p>This won't make anything more incremental, but when this is in place we can start splitting these larger queries into smaller incremental ones.</p>



<a name="156615787"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156615787" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156615787">(Jan 22 2019 at 16:02)</a>:</h4>
<p>So <code>hir_map</code> would be a <code>no_hash</code> query and then there would be additional, regular queries for specific lookups?</p>



<a name="156615808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156615808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156615808">(Jan 22 2019 at 16:02)</a>:</h4>
<p>(kind of like the codegen_units query?)</p>



<a name="156616226"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616226" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616226">(Jan 22 2019 at 16:07)</a>:</h4>
<p>@mw No. We'd just use the HIR map as is (when returned by the query which creates it), as it does it's own dependency tracking.</p>



<a name="156616585"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616585" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616585">(Jan 22 2019 at 16:12)</a>:</h4>
<p>Yeah, that would work too. But it would be nice to get rid of the special casing in the medium term.</p>



<a name="156616686"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616686" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616686">(Jan 22 2019 at 16:13)</a>:</h4>
<p>hm, although, how would that work?</p>



<a name="156616787"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616787" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616787">(Jan 22 2019 at 16:14)</a>:</h4>
<p>a query would do something like <code>let hir_map = tcx.hir_map();</code> and then <code>hir_map.expect_item(...)</code> or something like that.</p>



<a name="156616837"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616837" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616837">(Jan 22 2019 at 16:15)</a>:</h4>
<p>but <code>tcx.hir_map()</code> is a <code>no_hash</code> query, so every query calling it would always be re-evaluated.</p>



<a name="156616979"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156616979" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156616979">(Jan 22 2019 at 16:16)</a>:</h4>
<p>or maybe I am misunderstanding something?</p>



<a name="156617818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156617818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156617818">(Jan 22 2019 at 16:28)</a>:</h4>
<p>No query will actually call <code>hir_map</code> though, they'll use <code>tcx.hir()</code> instead, which calls <code>hir_map</code> in an ignore context, so no dependency to the <code>hir_map</code> query will be added</p>



<a name="156627690"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627690" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627690">(Jan 22 2019 at 18:36)</a>:</h4>
<p>wait what :)</p>



<a name="156627735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627735">(Jan 22 2019 at 18:37)</a>:</h4>
<p>why do we need an ignore context exactly?</p>



<a name="156627741"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627741" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627741">(Jan 22 2019 at 18:37)</a>:</h4>
<p>you could also imagine that the hir-map is just treated as 'always dirty'</p>



<a name="156627747"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627747" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627747">(Jan 22 2019 at 18:37)</a>:</h4>
<p>(right?)</p>



<a name="156627755"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627755" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627755">(Jan 22 2019 at 18:37)</a>:</h4>
<p>so that it is always constructed?</p>



<a name="156627811"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627811" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627811">(Jan 22 2019 at 18:38)</a>:</h4>
<p>is that the same thing?</p>



<a name="156627823"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627823" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627823">(Jan 22 2019 at 18:38)</a>:</h4>
<p>(in general, I'd definitely prefer to have fewer "exceptions" where possible)</p>



<a name="156627856"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627856" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627856">(Jan 22 2019 at 18:38)</a>:</h4>
<p>If there's no change to the crate, the HIR map will be green, but still executed since we don't save the result</p>



<a name="156627861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156627861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156627861">(Jan 22 2019 at 18:38)</a>:</h4>
<p>I'm not entirely convinced the HIR map makes a ton of sense, but it seems like if we move the HIR map methods to queries, we can avoid having any exceptions at all, no? (except maybe a "volatile" flag on the base map, purely for efficiency and/or to avoid adding impls / serialization?) maybe it results in some overhead, that is the fear?</p>



<a name="156628083"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156628083" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156628083">(Jan 22 2019 at 18:41)</a>:</h4>
<p>We need an ignore context since any change to the crate will result in the HIR map query dep node being red. Any queries which has a dependency on it will need to be executed again.</p>



<a name="156629135"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629135" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629135">(Jan 22 2019 at 18:55)</a>:</h4>
<p>that seems like it's sort of "by design"</p>



<a name="156629143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629143">(Jan 22 2019 at 18:55)</a>:</h4>
<p>that is, most things should not <em>directly</em> interact with the HIR map</p>



<a name="156629150"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629150" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629150">(Jan 22 2019 at 18:55)</a>:</h4>
<p>but rather queries on the HIR map</p>



<a name="156629163"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629163" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629163">(Jan 22 2019 at 18:55)</a>:</h4>
<p>but ok I see, you are aiming not to modify the code that is doing <code>tcx.hir().foo()</code></p>



<a name="156629217"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629217" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629217">(Jan 22 2019 at 18:56)</a>:</h4>
<p>I feel like I'd rather that the HIR map as an entity goes away, at least eventually, and we just move to <code>tcx.foo()</code></p>



<a name="156629304"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156629304" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156629304">(Jan 22 2019 at 18:57)</a>:</h4>
<p>I am aiming to not modify existing code and also avoid the performance overhead of adding queries which just forward information from the HIR map.</p>



<a name="156637282"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156637282" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156637282">(Jan 22 2019 at 20:37)</a>:</h4>
<p>it feels like those queries potentially play an important role, though, in terms of helping to propagate changes</p>



<a name="156637363"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156637363" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156637363">(Jan 22 2019 at 20:38)</a>:</h4>
<p>right now, at least IIRC, we have special dep-nodes for sub-parts of the HIR map, basically, but we could sort of remove that whole concept and instead just be talking about the results of queries</p>



<a name="156637428"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156637428" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156637428">(Jan 22 2019 at 20:39)</a>:</h4>
<p>(but cost is obviously a concern, we may indeed want some special treatment for optimization purposes -- though I feel like I'd sort of prefer if these still "feel like" queries in the end, just specially implemented ones)</p>



<a name="156676100"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156676100" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156676100">(Jan 23 2019 at 09:47)</a>:</h4>
<p>it might be workable temporary solution but I really want to avoid the hir map having to duplicate all the dep-tracking and hashing logic.</p>



<a name="156676178"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156676178" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156676178">(Jan 23 2019 at 09:49)</a>:</h4>
<p>regarding existing code: <code>tcx.hir()</code> could return small wrapper struct that provides the interface of the hir map but forwards to regular queries.</p>



<a name="156676974"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156676974" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156676974">(Jan 23 2019 at 10:04)</a>:</h4>
<p>I imagine it things to look kind of like this: there's an <code>[no_hash]</code> <code>hir_map()</code> query that does all the mapping for the crate. then there is a regular <code>item_hir_map(DefIndex)</code> that returns a map from <code>LocalId</code> to the actual HIR. and then there's the mentioned wrapper behind <code>tcx.hir()</code> that keeps providing the existing interface. Something like that. There might have to be a separate query for bodies?</p>



<a name="156866819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156866819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156866819">(Jan 25 2019 at 17:05)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> I've been thinking about Lark's "fast" parse that was described in the video and was wondering how we might apply the laziness in our case and how expansion comes into play</p>



<a name="156866902"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156866902" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156866902">(Jan 25 2019 at 17:06)</a>:</h4>
<p>I imagine we could do a fast parse without doing macro expansion to get a guaranteed, but incomplete set of 'definitions' per given file but when we get to a name resolution on a definition not on that list (meaning it must've been produced during expansion) we might then try to macro-expand a given file and cache any intermediate results?</p>



<a name="156866958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156866958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156866958">(Jan 25 2019 at 17:07)</a>:</h4>
<p>(I hope this topic is good for general idea/discussion and not specifically the existing query system details <span aria-label="grinning" class="emoji emoji-1f600" role="img" title="grinning">:grinning:</span> )</p>



<a name="156875813"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156875813" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156875813">(Jan 25 2019 at 19:06)</a>:</h4>
<p><span class="user-mention" data-user-id="153740">@Igor Matuszewski</span> applying to Rust definitely seems harder in some ways. Something I hadn't fully appreciated before is how having had an "IDE-focused compiler" earlier on might have affected the language design in various ways -- closures, impls, and nested items are all examples of things I might have done differently.</p>



<a name="156881259"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/156881259" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#156881259">(Jan 25 2019 at 20:15)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> <span class="user-mention" data-user-id="116466">@Zoxc</span> so I was thinking more about this. I agree that as a temporary measure, what <span class="user-mention" data-user-id="116466">@Zoxc</span> proposed makes sense, but for a longer term plan I definitely think we should not be doing anything "fancy" but just leveraging the incremental system.</p>
<p>I am not convinced we want a <code>hir_map</code> query at all longer term. We should see if we can make HIR lowering more "targeted" -- e.g., I think you can get the HIR for a struct without having to also create the HIR for its sibling items.</p>



<a name="157028495"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157028495" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157028495">(Jan 28 2019 at 14:28)</a>:</h4>
<p>I guess that we always have to lower an entire "hir owner" at a time so that we can properly assign <code>LocalId</code>s?</p>



<a name="157033112"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157033112" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157033112">(Jan 28 2019 at 15:30)</a>:</h4>
<p>I would be OK with doing <span class="user-mention" data-user-id="116466">@Zoxc</span>'s proposal as an intermediate step, but I'd really want to get rid of the manual dep-tracking for the HIR soon thereafter.</p>



<a name="157033175"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157033175" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157033175">(Jan 28 2019 at 15:31)</a>:</h4>
<p>I'd prefer to even skip this intermediate step but that might mean maintaining a branch for the refactoring for quite a while.</p>



<a name="157033430"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157033430" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157033430">(Jan 28 2019 at 15:34)</a>:</h4>
<p>I think we should remove NodeIds before trying to split the HIR map up. We also need some way to deal with non-item local things like <code>impl Trait</code>.</p>



<a name="157038584"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157038584" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157038584">(Jan 28 2019 at 16:43)</a>:</h4>
<p>Agreed, it’d be good to move <a href="https://github.com/rust-lang/rust/pull/57578" target="_blank" title="https://github.com/rust-lang/rust/pull/57578">https://github.com/rust-lang/rust/pull/57578</a> over the finish line and merge</p>



<a name="157056194"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/157056194" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#157056194">(Jan 28 2019 at 20:48)</a>:</h4>
<p>removing <code>NodeId</code> and other such refactorings makes sense</p>



<a name="163109424"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/163109424" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#163109424">(Apr 11 2019 at 15:21)</a>:</h4>
<p>I found yet another reason to want my end-to-end queries PR. I should make a to do list of things blocking on it</p>



<a name="163210592"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/end-to-end%20queries/near/163210592" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/end-to-end.20queries.html#163210592">(Apr 12 2019 at 17:40)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> actually, that would be a good idea :) (a list of things blocked etc)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>